EvalIta 2011: The Frame Labeling over Italian Texts Task

The FLaIT task held within the EvalIta 2011 challenge is here described. Systems were asked to label semantic frames and their arguments as evoked by input predicate words over plain text sentences. Proposed systems are based on a variety of learning techniques and achieve very good results, over 80% of accuracy, in most subtasks. 1 The Frame Labeling Task In the “Frame Labeling over Italian Texts” (FLaIT) task, the general goal is to come forward with representation models, inductive algorithms and inference methods which address the Semantic Role Labeling (SRL) problem. This is the first time that such a task is proposed in the framework of the EVALITA campaign. So far, a number of shared tasks (CoNLL–2004, 2005, 2008, 2009 and Senseval/Semeval–2004, 2007, 2010) have been concerned with SRL. Typically, two main English corpora with semantic annotations from which to train SRL systems have been used: PropBank [3] and FrameNet [1]. These previous experiences have been focused on developing SRL systems based on partial parsing information and/or increasing the amount of syntactic and semantic input information, aiming to boost the performance of machine learning systems on the SRL task. Since 2009, CoNLL has been accompanied by a shared task dedicated to SRL not restricted to a monolingual setting (i.e. English) [2]. The Evalita 2011 FLaIT challenge is the first evaluation exercise for the Italian language, focusing on different aspects of the SRL task over Italian texts. The interest in organizing this challenge has been prompted by the recent development of FrameNet–like resources for Italian that are currently under development in the iFrame project. 1 http://verbs.colorado.edu/∼mpalmer/projects/ace.html 2 https://framenet.icsi.berkeley.edu/fndrupal/ 3 http://sag.art.uniroma2.it/iframe/doku.php